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1. INTRODUCTION 

1.1 The Task of the Paper 

In this paper we outline a framework for a program of studies to 
assess the impact of the Federally funded Research and Development Center 
program on American education. Such an assessment is a task of great 
magnitude. Inevitably, it will, necessitate several separate investigation? 
conducted by different investigators. These studies must both logically 
relate to one another as well as yield a comprehensive picture of the 
impact of this major aspect of Federal science and education policy. 
This preliminary problem analysis and design paper can be the groundwork 
for assuring such coherence and comprehensibility. We present, therefore, 
an analysis of the scope of the problem and make suggestions for research 
approaches; we describe a strategy for a research program, but we do not 
offer designs for the component studies. 



1.2 Approach 

Assessing the impact of a major and complex policy like the 
Research and Development Center Program (which includes both the Researcl 
and Development Centers and the Regional Laboratories), is a difficult 
and important undertaking. Neither science nor education policies are 
knowr as areas that lend themselves to easy evaluation by direct 
inspection. The Research and Development Center Program is particularly 
complex because its introduction coincided, and possibly caused, a major 
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restructuring of orgeuiizations and priorities in the field of educational 

research. Therefore, the plan to measure the impact of Federally financed, 

disciplined research and development on education requires sone systematic 

reflection on the major domains which were affected by the policy, and 

■ 

on the criteria by which they are to be assessed. 

The Federal policy modified an already existing and extremely 
complex set of nationwide and local arrangements for the production, 
distribution, implementation, and use of educational knowledge."^ It 
would be misleading to measure the merits of the effort merely by measuring 
direct effects on school practice, as they are presently obtained in the 
field. Because of the complexity of the systems for knowledge production, 
distribution, implementation, and use in American society, systematic 
leewis and lags must be anticipated as innovations are spread through the 
system, and major impacts may be of the unanticipated variety. 

In this design paper we, "cherefore, follow a systemic approach 
to impact assessment. Premature commitments to simple indicators should 
be avoided. For example, measuring impact and effectiveness only through 
adoption rates of products, or measurement of the cost of adopted products, 
r.ay not only be misleading but also may have the harmful effect of pressing 
for shori: range results even in situations that demand long range 
solutions. 

The perspeccive in terms of which the design pAper is written is, 
then, entirely consonant with the approach taken by the staff of the 
r.'ational. Institute of Education in their December 19T3 report on knowledge 
prcduction and utilization in education,"' and with the rather comprehensive 
•"■iew taken in the earlier study prepared by the Bureau of Research in the 



Office of Education,^ which presented to the Organization for Economic 
Cooperation and Development a report on educational research and development 
policies in the United States. 

1.3 Utilizability; Relation to System Monitoring 

The design of the plan to assess the impact of the research and 
development system should be such that the impact study itself can become 
the basis for a longer range program of monitoring the system's performance. 
In the specific design efforts of component studies it will be necessary 
to investigate alternative impact indicators. It may be an economical 
procedure to use this work as the base for a program of system monitoring 
in which these indicators would be periodically assessed. The initial 
design procedures must therefore be flexible, and must 6lL1ow for inputs 
both from the potential users and the producers of the information to be 
used in the monitoring program. We attempt to make provisions for such 
flexibility and usability. 

The execution of the impact studies program is a large enterprise. 
A design consideration for which we are in no position to make specific 
recommendations, but which should be weighed, concerns the timing of 
impact studies in relation to the timing of policy decisions to be made 
in the Institute. The development of an integrated schedule may facilitate 
both the conduct and the utilization of the research. 



Z. DESIGN CONSIDERATIONS: IMPACT DOMAINS AND IMPACT PATTERNS 
2.1 Introduction; Policy Goals 

The Research and Development Center Program was established, 
approximately ten years ago, in order to speed the process of generating 
and applying scientific knowledge to education for the purpose of improving 
instructional practice and programs.** This policy was the second phase 
of major Federal research support for educational innovation and knowledge 
production; the earlier phase of grant support for individual projects 
was initiated approximately twenty years ago. It would appear that the 
decision to adopt the Research and Development Center Program policy was 
based on the following considerations: it was felt that the-^jsaxlier 
pattern of supporting and coordinating educational research through 
individual, grants did not lead to an adequate concentration of resources 
and talent for the kinds of sustained effort that could reasonably be 
expected to produce ciimulative and conclusive results. Project research 
had remained small and fragmentary; there existed a large gap between 
research and practice; the field of educational research did not attract 
leading social and behavioral scientists; and there appeared to be a lag 
between basic social and behavioral science theory and methodology and 
the field cf educational research.^ 

The Research and Development Center Program thus was a policy 
conceived to overcome these problems by supplementing small scal^e efforts 
with large prograns of interconnected research and development. The 
strat^g^' to assemble a "critical mass" of research scientists who would 



direct their efforts towards applied research and development of educa- 
tionally useful products was not Just intended to produce specific 
innovations but also to impact the field of educational research by 
providing structure, leadership, and new models and standards of Judgment.^ 
The policy is an ambitious one; it is designed to impact the entire 
system of educational knowledge production and use in a strategic manner. 

The key conceptions in this enterprise appeared to be the notions 
of scientifically disciplined research and development, and of scientifically 
grounded, codified professional practice in schools. The assumption is 
that only scientifically disciplined research leads to cumulative knowledge 
and the detection and correction of errors; scientifically grounded, and 
disciplined practice would be necessary to improve the performance of 
schools."^ Since the program has such broad policy objectives, it is 
reasonable to meaaure its impact over several domains. Considerations 
concerning the availability and mobilization of manpower, the development 
of organizational structures, and management systems, the scientific stock 
of knowledge, technologies and procedures, and standai'ds of p -ofessional 
practice need to be considered across the target domains. 

2.2 Impact Domains 

Four empirically and analytically distinct domains can be dis- 
tinguished in any large scale, institutic. .alized system utilizing 
scientific knowledf^e for the improvement of professional practice. They 
are the systems of knowledge production, of knowledge distribution, of 
knowledge implementation, and of knowledge utilization or use, in ongoing 
activities . 



It is fairly clear that the dynamics of knowledge production in 
research establishments and elsewhere differ sharply from the processes 
of knowledge distribution through markets or organizational channels. 

Q 

This distinction has been well recognized. However, it is less often 
recognized that the domain of implementation of programs based on new 
knowledge differs both from the knowledge distribution and use of systems. 
A brief remark concerning our insistence to deal with implementation as a ^ 
distinguishable system may be necessary. Implementing a new process 
almost always leads to an at least initial increase in complexity, and 
in higher dem;inds on time and effort -of staff, and thus to organizational 
overloads. Since such episodes of implementation occur with some regu- 
larity in knowledge utilization systems, there develop more or less 
institutionalized arrangements for dealing with these problematics. The 
"implementation system" includes those organizational arrangements and 
aspects of institutions which offer supports of various kinds, such as 
funds, personnel, advice, to reduce the temporary overload and facilitate 
the rout inizat ion of innovations into the use system 

The "use system" refers to codified, professional practices in 
ongoing operations, into which new knowledge or programs have become 
incorporated. 

2 . 3 Ir.pact Patterns and Criteria 

rt'e distinguish between different patterns of impact which may be 
-.raced thi'ough alternative approaches. In principle it appears straight- 
rorwari to assess the impact of any given Research and Development Center 
by Icca'.ing instances of adoption of their products (and such, empirically 



quite complex, work certainly needs to be done). However, this approach 
would not in itself assess the impact of the policy to promote the Research 
and Development Center movements in terms of strategic significance for 
the -structuring of the field of educational research knowledge and 
application. The former problem requires studies of product distribution 
and use, which should be conducted both by tracing the distribution of 
innovations from their source to the site of use, and in the reverse 
direction. The latter problem requires c^n assessment of the organization 
of the field, the effect on manpower supplies and mobilization, and the 
relationship between R&D center priorities and those in the field 
generally. A major issue in this regard is the spread of the standards 
associated with disciplined research and development through Americans 
educational institutions. The search for such an impact pattern certainly 
will require an approach somewhat different from a study of product 
diffusion. 

We therefore propose that studies be done in the four impact 
domains (knowledge production, distribution, implementation, use), in 
their linkages with each other, tracing impact patterns both from the 
knowledge production sector to the domain of utilization, and from 
operating school systems to Ihe source. 

These ccmplem' .itary approaches will shed light on ^he relative 
significance of the knowledge produced in the Research and Development 
systen in relation to" the total volume of knowledge used; they must be 
supplemented by the study of the scientific movement spearheaded by the 
Re:::earch and Develcpr.ent Centers and its impact on the total field. It 
is in this latter area that strategically important, long range impacts 
n-ay oe developing. 



The question of which effects are to te considered significant 
impacts, and how they ought to be causally attributed to the policy is 
to be treated in the specific design phase of each study; we limit our- 
selves in the following to outlining the process in which specific 
determinations of impact criteria and attributions appear in principle 
feasible, and we make some suggestions in this direction; we are guided 
in this by the stated objectives of the policy as they are discussed above. 

In the next sections we outline the issues to be studied in 
relation to each impact domain and linkage area; this is followed by 
a section outlining a system of interlocking studies to obtain this 
information. 



3. THE KNOWLEDGE PRODUCTION SYSTEM 

3#1 Ma^or Questions 

The central issues with regard to the knowledge production system 
are simply: 

~ What is the position and relative significance of the R&D Centers' 

programs in the total educational knowledge system? 

-~ In what ways have the Research and Development Centers' programs 

changed the production of educational knowledge? 

— • What are the consequences of these changes? 

The treatment of these issues requires a combination of historical and sociology 

of science approaches. Historical reconstruction will not be difficult since 

several good studies of the state of affairs in the early sixties are 

9 

available and may serve as a base line. However, it appears to us that 
many of the issues outlined in systematic overview fashion below have not 
been raised and that therefore new information gathering activities will 
be necessary. 

3 . 2 Mapping the Domain and its Structure 

Obviously » the Research and Development Center Program does not 
exhaust the domain of educational knowledge production in the United 
States. A current overview of organizations engaged in educational 
research may, for a firsc orientation, be derived from an analysis of 
available organizat iona2 directories."^^ The major types are: the 
university basei research and development centers; other university based 
educational research institutes; the Regional Laboratories; non-profit 
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research establishments like the American Institutes for Research; profit 
making research corporations; publishing houses; research bureaus in 
state departments of education, in large school districts, and so on* 
Trends in these organizational patterns over the ten year period of the 
R i D policy can be established and measures of relative productivity may 
be provided. These trends may or may not reveal the realignment of organiza- 
tional patterns in response to the establishment of the Research and 
Development Centers. 

Particularly significant patterns of such knowledge producing 
organizations may be identified for more detailed review with the objective 
of determining their relative significance and location in the total system. 
For example, the large non-profit institutes have made well known con- 
tributions of major significance ;"^"^ any review of the* organizational 
structure of the field would have to pay particular attention to this 
factor. 

Yet, educational knowledge is not only produced in the organizations 
deliberately designed for this purpose. It is also created in social 
movements, in public debates, through literary efforts, and through local 
innovations by expert practitioners.-^^ Much of this "knowledge" is not 
being presented in the model of scientifically rigorous research and 
development; but any inspection of the American educational scene reveals 
that such political and intellectual trends, local innovations, the efforts 
of activist groups and the like have a major impact. In fact, it is not 
a settled question whet!inr the implementation and use systems are more 
affected by this sector or by the formal organizations of professional 
education research. In order to map the total system it is important, no 
rr.atter how difficult, to arrive at an estimation of the scope, stinicture, 
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and direction of these activities to permit a look at the relationship 
between federally sponsored and formal and other k$iowledge production* 
Interview, or assessment symposia^ with highly informed and articulate 
persons may be one avenue for an economically feasible approach to the 
problem. Education editors of major newspapers and magazines come to mind 
in this connection. 

3.3 Standards 

The leadership of the Research and Development Centers can be 
considered a movement for reform in science and educational practice. They 
advocate a standard of "rigor" in applying scientific concepts and methods 
in education which appears to give the work and possible impact of these 
organizations a particular quality and direction. For example, it is 
demanded that innovations in education be designed following scientifically 
grounded principles, that they be pre-tested, systematically developed, 
and evaluated, and that under no circumstances are they to be simply 
introduced on the strength of enthusiastic commitment . This emphasis 
on fairly specific conceptions of standards differentiates this reform 
movement sharply from other efforts to improve contemporary Americetn education, 
as for example in the "free schools." Similarly disciplined and profession- 
ally informed behavior is also expected of the teacher, whose "art," 
a::cording to the research and development movement, is to be enhanced 
by a broad repertory of scientifically grounded tools. 

Certainly, the movement to utilize behavioral science and to 
codLCj prcfeGsioaal practice in education is not simply cc~extensive with 
tr.e Research and Development centers. However, there appears to exist 
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toth affinity and a major overlap of personnel. It should not be 
difficult — and it would be highly informative — to gather evidence that 
has a bearing on these points. To do so would be helpful for impact 
assessment and policy formation since the successful propagation for 
standards of professional practice may have effects of a profound and far- 
reaching nature. 

Empirical questions to be answered might include the following 

issues : 

Tp what extent is there consensus among Research and Development 

Center leading personnel on standards of professional work and on 
what constitutes "disciplined research and development"? 

— To which sources and groups can the commitment to the standard 
be traced? Professional leaders of the field of educational 
research advocated the model of scientifically disciplined efforts 
well before the adoption of the policy by the government; 
however, the emphasis was not uncontroversial. To what extent has 
the policy reinforced a circumscribed intellectual elite ia their 
positions, and to what extent has there been an exercise of leader 
ship by demonstrating to the field at large the need for the 
advocated standards and their superiority over alternative models? 

'^-hat is the structure of the social networks among the research 

and development leadership and among the productive persons in 
the field of educational research in general? Of particular 
i.TXcrtance in this regard is not only to map the networks or 
"invisible" colleges, but also to assess the prestige structure 
and the position of the relevant individuals and groups within 
it. It is not recommended tc use a study of prestige ratings 



simply in order to determine "quality," as this might be quite 
misleading in a rapidly developing field, but to shed light on the 
direction of leadership and the persuasiveness of standards 
advocated. 

— Has there been €Ln effect of the demand for rigorous standards on 
publications ajid other policies in the scholarly associations in 
the field of educational research, as demonstrated by the nature 
of review procedures, the composition of review panels, the 
structure of scholarly programs at meetings, and referral networks 
for personnel recruitment? 

— • Have there been effects of these processes on the curricula of 

schools of education and of other graduate departments in related 
fields? 

The question of whether a possibly detected change in the standards 
of the wider field of educational research and related disciplines can 
be attributed to the research and development centers may be tackled 
indirectly through mapping the position of research and development center 
personnel in the structure of social network and prestige structures among 
education researchers, in scholarly associations and organizations, and 
through investigating the history of diffusion of model standards for 
disciplined research and development. 

3 . ^ Priorities 

As in any scientific enterprise, priority setting for the knowledge 
production efforts of the Research and Development Centers is a complex 
process. The nature of this process, however, can be clarified by 
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investigating the frames of reference of research and development leaders, 
and the strategies they employ for the detection, priority assignment » 



and solution of problems • Pi*o>i>am detection, and the assignment of priorities 

' -A 

in the distribution of efforts] to tackle them, is, probably, the single 

V 

most central process in knowledge production. The criteria and structure 



of assumptions involved in these processes is often implicit and unreflected 
leading investigators to the discovery of "interesting'' or "fascinating" 
issues indirectly. However, the sources of such problem detection processes 
can be investigated by studying the structure of frames of reference, and 
the cognitive strategies for problem identification and solution. It is 
particularly important in understanding a research and development enter- 
prise to provide knowledge about the sources of investigative problems 
chosen. Tl^fese would include disciplinary paradigms and theories, concern 
with public issues or value conflicts, a commitment to problem solving in on- 
going professional practice as well as other sources. It is to be expected 
that the professionalization a nd increasing complexity of the structure of 
the field of educational rerearch may lead to a shift of the sources of 
investigative problems into the internal dynamics of the professional 
domain itself. As a consequence there may be a divergence from problem 
detection processes in operating educational systems and public debates. 
Such divergencies or convergencies may have subtle, but in the end, major 
effects on the acceptance of products, and indeed, of the research and 
development effort. 

Specific priority setting mechanisms can be investigated by 
studying the reference frame of research and development leaders through 
detailed interviews, the analysis of their research strategies, the 
utili::aticn of programmatic statements, and the relation of these findings 
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to the actual distribution of effort in the organizations. Comparative 
work with non^research development persons in the knowledge production 
and in the distribution, implementation and use domains is necessary. 

The pattern of priority setcing in the Research and Development 
Program may have changed over time. It is at least possible that in the 
early phases of the program priorities were assigned to projects expected 
to yield at least some visible results in a short time. It is further 
likely that monitoring and auditing procedures used by the government 
sponsors would and continue to have an effect on priority setting. It 
is important to discover these trends and the relationship between priority 
setting and governmental evaluation of the centers. Such information 
needs to be taken into account in designing the program monitoring of the 
Research and Development system in the future. 

3.5 Stock of Knowledge 

Educational research, including some of the work carried on in 
research and development 'centers, contributes to various bodies of 
knowledge for practicing educators, for educational research itself, and 
for certain scientific disciplines. An assessment of the structure of the 
stock of knowledge in use in the centers, their contributions to it, and 
their contributions to other fields would be required if one wishes to 
show their impact in this area. Both qualitative and quantitative approaches 
seem indicated. 

Several "state of the art" assessments of the available . stock of 
knowledge in educational research have been undertaken before, for example 
in annual reviews of educational psychology, in various presidential 
addresses before the Aiaerlcan Educational Research Association and other 
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scholarly bodies, and on other occasions.-*-' A major overview may well be 
fruitful at this time since many investigators are claiming the occurrence 
of a "paradigm shift" from behavioristic learning theory as the basic 
scientific foundation to cognitive psychology and social psychology ."^^ 
Such work might be conducted, for example, by the National Academy of 
Education in analogy to their overview "Research for Tomorrow's Schools." 
A quantitative study of research production and of the network of citations 
may be helpful in locating the research and development centers within the 
knowledge production system and demonstrating their contribution to the 
working stock of knowledge of the field. 

3*6 Products: Devices, Technologies > Procedures 

Considerable emphasis has been placed on the assessment of Center 
productivity by measuring the effectiveness of their products and their 
distribution. For example, the review conducted by the Comptroller General 
of the United States and submitted to the House of Representatives and 
Senate in August 1973'^'^ measures the impact of the R&D centers almost 
entirely through a rather searching overview of production evaluations and 
marketing and dissemination strategies. In the impact research program 
recommended here, such efforts also are considered of major importance. 
However, as the foregoing has made clear, they ought to be imbedded in a 
strategy to map the total impact domains. , 

In order to trace Center products it will be necessary to construct 
a careful list of products and their characteristics. This may appear 
as a simple and straightforward task, but it is not. It is insufficient 
to review Center reports and cull from them the names of innovative programs 
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that have entered the dissemiaation process. Tracing the distribution 
of products by tracing the acquaintance with or utilization of their names 
is only a first, and superficial, approach. More appropriate is th^ 
detection of key tracer "tags" in the developed educational technologies 
and procedures so that actual practices might be identif ied» \ Such a list 
of products and their characteristics for the purpose of tagging traceable 
components requires detailed analysis of the matter by technically qualified 
personnel. 

Estimates of produc ^ costs should be undertaken with due accounting 
for the .costs of other desired impacts. 

For each major product a degree of utilizability assessment might 
be made, including a review of the product *s legal status (for example, 
in relation to copyright laws), and its relation to the standards, 
priorities, and theoretical principles of the field. 

With this material provided, it is possible to construct a macro- 
analysis of the institutional channels for product distribution, implementa- 
tion, and use through a survey of key personnel and the scrutiny of reports. 
It is not recommended to conduct such a study in great depth or in detail, 
but to supplement it by tracer studies following selected products from 
knowledge production through the system to use. Such tracer studies, 
taking as their point of departure the knowledge production sector in the 
Centers, should be supplemented by case studies described below in the 
distribution, implementation, and use systems. Such work will be 
facilitate i by integrating the studies with the available evaluative 
studies of product effectiveness. 
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3.7 Organizational Structures and Nesting 

Research and development in education has, over the period of the 
last decade, been groping for appropriate organizational and management 
models. An overview of alternative models for organizations that have 
been generated in the R&D Cent-:*r Program, supplemented by in-depth case 
studies of selected centers, could be extremely helpful in improving 
leadership and management procedures, and in demonstrating whether the 
Centers have produced organizational innovations of their own. 

The research and development centers are located in universities, 
but in different nestings. The dif ferent.ial impact of such nesting 
arrangements, for example in schools of education or in alternative 
locations in the university structure, would give initial data on the 
impact of the research and development centers on their host universities. 
Since "their location in universities may affect the instructional practices 
of these institutions, this impact domain must not be neglected. 

The regional laboratories, being of a different organizational 
design, may be expected to have evolved certain novel conceptions of their 
own in the area of organizational structure and management procedures. 
A similar approach of a broad survey and specific case studies could be 
followed for both types of organizations. 

It should be noted that the study of problems mentioned earlier 
in relation to the effect of the research and development program on 
scientific fields and their scholarly associations would shed light on 
the organizational impact of the center program on these fields. 
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3*8 Manpower 

The availability of highly qualified manpower for educational 
research and development has been a problem for some time. It can "be 
expected that the existence of research and development centers and the 
educational regional laboratories has had a major effect on the situation 
in this atrea. An investigation of the effect of these orgeuaizations on 
the structure of channels for social mobility of persons entering careers 
in behavioral and social science and educational research is much needed. 
It is therefore recommended to supplement quantitative assessments of the 
available manpower pool in the relevant training areas by studies of careei* 
channels and patterns » including studies of career aspirations of graduate 
students. It may also be the case that the necessity of recruiting personnel 
for research and development work from schools and school administrations 
and other professional domains may significantly impact the. career patterns 
of these persons and thus have an effect on the manpower pool available 
for educational innovation more generally. Estimates in this domain would 

require investigations of subsequent career experiences of persons who for 

. *^ 

a time worked in R & D Centers or Regional Laboratories, and the effects 
\ 

that can be attributed to their work in the program. 
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U. THE KNOWLEDGE DISTRIBUTION SYSTEM 

\ 

t 

k.l Ma.lor Questions 

The main concerns with the knowledge distribution system in 
edaca^-Lon are; 

— What is the position of the research and development centers and 
educationad laboratories in relation to the total educational 
knowledge distribution system? 

— How effectively are research and development knowledge products 
disseminated? 

— What are the filtering effects of the system, what are its 
conductivity characteristics? ^ 

— In what ways have the research and development tenter program 

and the laboratories impacted the educational knowledge distribution 
system itself? 

k.2 Mapping the Domain and its Structure 

The system through which eduoational knowledge is distributed 
comprises a vast, and for the most part, loosely structured network of 
conununication channels. Certain components, of course, stand out: the 
commercial market of educational materials with its advertising efforts; 
trend setters and gatekeepers among publicists and the media; the arena of 
educational issues and debates in the public domain which include the 
channels of mass media, magazines, trade books, and the efforts of interest 
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groups; informal, interpersonal networks among experts and practitioners; 

organized knowledge availability systems, such as ERIC; the communication 

channels of professional and scholarly societies, such as meetings. 

Journals, handbooks; especially designed efforts to distribute educationcQ. 

knowledge through deliberately created organized channels, as for example 

in certain efforts of the regional laboratories, and others* 

How can one reasonably expect to describe such a large and varied 

domain and obtain usable results? In the first place, previous studies 
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have clarified part of the area, and the systematic assessment of the 
literature for the purpose at hand will be profitable* 

Secondly, an overview of the total system, its differentiated 
conductivity and its tutoring effects, can be sketched by utilizing the 
knowledge of expert professionals, such as prominent education generalists. 
Thirdly, a limited scope research progreun, tracing the structure of the 
knowledge distribution system from both the input and output side will be 
necessary. The questions here axe: 

W.at does the system "deliver" to the practicing educator? 

— How ioes ;:. ican the domain for knowledge? 

— . Ho. is in J search behavior affected by the differential conductivity 
and 'rtering effects in the system? 
On the ot/r.;r J, thi re arise the issues: 

— Wh:i: iti l .ue cf 1 esearch a.id development center inputs into 
tho kno /ledge dlGtribution system? 

"racer studies from both sides are recommended as a procedure to determine 
t:>? r.r,t terns of .11 ff'erential conductivity in the knowledge distribution 
syster*. 
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l+,3 Standards 

The dissemination of the standards of professionalism advocated 
by the -research and development movement is one of its desired impacts. 
However, the explicit conception of educational reform through science 
and technology competes with several alternatives in the arena of 
educational change. One would want to know a great deal about the dis- 
tribution of preferences for what is needed in educational change among 
the significant actors in the knowledge distribution system. What, for 
example, are those sectors of the system that have a particular affinity 
for the research and development approach and its products; are there 
sectors that have resistances to this approach? 

The question of the distribution of standards, then, has two 
aspects to it. There is the differential distribution of knowledge about 
and acceptance of the model of educational professionalism described above 
and the distribution of basic paradigms for education, that is the images 
•of the child as learner and of what "high quality" education means. 

It is reasonable to suppose that the distribution of beliefs about 
these matters among actors in the knowledge distribution system have been 
specifically impacted by the Research and Development Center Program. 
Further, it is likely that the social patterns of acceptance or rejection 
cf standards and paradigms may, in part, account for the differential 
conductivity of the system when it comes to the distribution of specific 
devices, techniques and procedures. 

In order to tackle this kind of question, it would be highly 
.iesirable to identify those persons who are significant gatekeepers in 
the knowledge distribution system of education, and to investigate their 
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characteristics, l^eliefs, and practices by means of a survey. Should 
there be practical knowledge needs toward the highly specific information 
to be obtained in this matter, the effort may be Justified, However, we 
recommend an approach of utilizing symposia and group discussions in 
order to obtain gross estimates of this information at lesser costs. 
Clearly, some reliable knowledge of ^he characteristics of knowledge 
distribution gatekeepers is a key item for planning effective knowledge 
dissemination efforts. One might expect in this arena certain cohort 
effects, effects of basic values and attitudes towards political issues. 

Some attention must be paid to the distribution of educational 
knowledge through the training and education of professional^. In this 
area we would expect to find relationships with the knowledge distribution 
system in the academic disciplines and in schools of education. The 
centers themselves have a training impact which is not negligible. Certain 
key concepts of theirs may well have found their way into graduate and 
xindergraduate curriciJila in universities. This matter might be seen as 
closely connected with the manpower aspects of knowledge production and 
should be investigated in that connection. 

k.k Priorities in Knowledge Demands and Supplies 

In analogy to the treatment of priorities in knowledge production, 
we can ask questions about the respective priority scales of those who have 
knowledge needs or demands as against those who regulate the supply. What, 
for example, are the knowledge and infomation needs of those who dis- 
tribute educational knowledge? What incentives operate on them (if any) 
to orient themselves towards the research and development system and its 
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products? What is the relationship between the priority structure of the 
knowledge demand » and the offering? 

The complex phenomenon of searches for reliable educationeLl 
knowledge by practitioners » and of the priorities for the information needs 
they perceive is not well understood • In. fact » the success of the Research 
and Development Program in distributing its output depends largely on its 
ability to generate a need for the knowledge it supplies in the distribution » 
implementation and use systems. It would therefore be most interesting 
to demonstrate what, if any, impact the Research and Development Center 
Program has had on knowledge demands for the results of educational 
research and development in schools with direct, indirect, or no formal 
relations with research and development centers and laboratories, and 
undertake the same with centrally placed persons or organizatloiis^in the • 
knowledge distribution system. ^ 

k . 5 Designs for Knowledge Distribution 

Several approaches to the problem of disseminating educational 
knowledge have been tried by the centers and laboratories. A summary 
statement of their experience can be prepared to assess the impact of the 
effort on the creation of appropriate models for knowledge distribution 
projects. This is largely an effort to gather and synthesize information 
by reviewing the experience of such projects, scrutinizing the reports 
and interviewing personnel. 
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k.6 Products; Devices t Technologies > Procedures 

On the basis of the detailed product list described above (item 
3t6), the scope of dissemination and the networks of product distribution 
need to be described. This enterprise has at least the following aspects: 

— An inventory and description of organized distribution networks, 
such as the programs constmicted in the "Follow-Through" effort, 
school networks of the laboratories and centers, and the like; 

a descriptive summary of the accumulated experience based on 
interviews with key personnel. 

Selected "tracer" studies of several products from centers to users. 

— * Utilization of a survey approach to study the scope of diffusion 

in the field, as well as the range of knowledge about center 
products. 

k.J Manpower 

Specialized roles have developed in the context of the knowledge 
distribution system, such as educational consultants, change agents, 
education editors and the like. The availability of quE*lified personnel 
for these responsibilities may have been affected by the efforts of lab- 
oratories and centers to disseminate and implement their innovations. 
This point links with the already identified need to review the impact 
of the Research and Development Center Program on the supply of educational 
manpower . 
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3. THE KNOWLEDGE IMPLEMENTATION SYSTEM 



5.1 Major Questions 

Innovations which reduce the complexity of the work process and 
increase the subjective certainty of the workers may find readier adoption 
and use> other things being equal, than innovations which, in the interest 
of increased effectiveness or other values, increase complexity and 
uncertainty.^^ Virtually all the innovations deriving from recent educa- 
tional research and development appeeur to be of the latter variety. It 
is this circumstance which produces special difficulties in implementation 
of newly developed programs and products. The special role of "prograjn 
implementor" has developed in some such settings. 

The major questions in this area are: 

— VTiat has been the effect of disciplined educational research 
and development on organizational arrangements, personnel, and 
policies for program implementation? 

— What conclusions can be drawn from the experience to date? 

5.2 Mapping the Domain and jts Structure 

A first overview of the domain of program implementation can be 
obtained by working outward from the resestrch and development centers and 
regional laboratories to identify explicitly organized implementation 
prc6.:ra4Ti£ . P-x. inventory of such na.lor implementation efforts may, in fact, 
be roGsible on the hasi3 of the perusal of report!' available in the NIE. 
This should be suprleir.ent.ed by interviews with experienced persons. 



Several programs and organizations operate in this field, A typology of 
such organizations and approaches can be constructed, which would provide 
inforoation important for deciding on a program of case studies. These 
studies would investigate the relation between implementation efforts and 
the use system, and the system's varying degrees of implementation capacity. 
Particularly complex, both successful and unsuccessful, implementation sites 
may be selected for analysis by the case study method. 

5.3 Standards' and Priorities 

This topic raises the dual -issue of standards for the assessment 
of implemented programs, that is the issue of the degree to which a 
program is actually being carried out as it is designed, and the question 
of professional standards for t^e implementation tasks themselves. 
" implement ors" even more than knowledge "disseminators" bridge the gap 
between knowledge and practice. They participate in at least two 
different noraatively regulated domains: the operating system of the school, 
and the sources of knowledge production and distribution. Priority 
settings, scheduling of activities, standards of judgment, in both areas 
are different. A study of the^e differences and of alternative impleipenta- 
tion strategies would be useful. 

5 . ^ Manpower 

It seems necessary to determine the characteristics and the supply 
of qualified educational pi'ogram implementors. Since the role is in an 
early phase of institutionalization, it will be difficult to accomplish 
this with any rigor. However, a possible approach might be to study the 
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research and development center implementation and school interface 
personnel, and to determine their backgroimd and career lines f ^ a first 
estimate of the sources for this type of personnel. Again, there is a 
linkage to other questions concerning the manpower training impact of 
the research and development centers raised before. 
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6. THE KNOWLEDGE USE SYSTEM 

6.1 Major Questions 

What is thoxight of as the "use system*' here are the schools of 
the nation I all the institutions in which formal education takes place. 
Clearly, this system of institutions not only uses knowledge produced 
outside it, but it also generates accumulated pedagogical, experience and 
expertise, knowledge of its own, and educational innovations. Here the 
major questions concerning the study of the impact of the Research and 
Development Center Program are: 

What is the relationship between innovative activities and 
knowledge production in schools, and the knowledge supply, 
especially as it is provided by the research and development 
system? 

— What alternative sources of educational knowledge are being 
utilized? Which are the major occasions which cause educational 
practitioners to search for applicable knowledge? 

— What actually happens in such searches? 

— Under what circumstances do they lead towards the Federal Research 
and Development Center Program? 

6.2 Mapping, the Domain and its Structure 

Here considerations arise which concern essentially the construction 
of an appropriate sample for the study of educational knowledge utilization 
in school districts, and in schools. It will be necessary to establish 



a sample of both school 'districts and individual schools for the purpose 
of investigating knowledge utilization* It must be recognized that this 
particular investigative purpose must be taken into account in structuring 
the sample itself* The task therefore will be quite complex in that the 
obvious criteria of region in the country, size of district, urban or non- 
urban location, ethnic and socioeconomic composition, and the like, need 
to be supplemented by major criteria for stratifying the sample in terms 
of relations to the knowledge production, distribution, and implementation 
systems. For example, schools and districts known to maintain direct and 
formal links to the research and development system need to be compared 
and contrasted with schools having no such ties* 

6*3 Standards 

Each of our ^alytically distinct systems, we think, can be 
described in terms of a prevailing pattern or distribution of standards 
of professional practice. It certainly would shed considerable light on 
the field to be impacted by the research and development center system if 
the social distribution of commitments to different conceptions of 
ctppropriate standards for responsible professional conduct were known, as well 
as the distribution of commitment to basic paradigms in pedagogical 
orientations . 

Earlier we discussed the proposition that one of the most far- 
reaching impacts of the Research and Development Center Program may well 
be the propagation of more rigorous standards of professional practice 
and new paradigms of pedagogy. It is at least likely that changes in 
pedagogical orientations and conceptions of what is desirable practice have 

30 



consequences of wide scope and possibly duration, which may surpass the 
effects of recently developed technological innovations in the field. In 
order to understand the mechanisms of such processes « the yays through which 
standards and paradigms in the teaching profession are shaped need to be 
clarified. For example, one might study the effects of cohorts, of 
schools of education, professional experience in the system, and other 
aspects by tracing them from schools to their source in the context of a 
large scale survey of schools and school districts. 

6tU Priorities 

The relative urgency with which knowledge demands and searches 
are pressed, or proffered innovations accepted, depends on the concrete 
constraints and incentives in the operating knowledge use system. At 
least it is reasonable to look in this direction for determinants. 
Teachers' priorities for knowledge obviously differ from those of research 
scientists, as they relate to a different set of professional responsi-- 
bilities and constraints. Relatively little is known about the priority 
structure for new educational knowledge among experienced teachers, 
principals, and other practicing educators. The explicit study of these 
priority structures from the point of view of the practical constraints 
shaping them also would be useful knowledge for planning and for counter- 
acting negative ideology formation among research and development personnel, 
concerning such conceptions as alleged ^^resistances to innovation" or 
"prejudices against science." 
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6.5 Stock of Knowledge 

What actually is the stock of pedagogical knowledge now in use 
in schools of different types? Obviously, some pedagogical theories and 
methods are in use in all schools; what they are and how the practicing 
profession maintains its stock of knowledge should be described. The 
issue is closely connected with our discussion of standards and priorities, 
but more specifically we are raising the issue of which pedagogical 
propositions, methods, and practices are known by teachers and other 
experienced professionals to be reliable. It should not be surprising to 
find that this practical stock of knowledge differs, maybe sharply, from 
what the leaders of educational research and development, and professors 
in schools of education advocate or accept; it is necessary to discover 
the grounds of knowledge validation in the practicing profession, and the 
differences between them and the conceptions of reliable evidence among 
researchers and developers. Such information has obvious relevance to 
ascertaining the mechanisms through which research and development center 
products may impact schools. It is likely that this information can be 
tapped in a large scale survey of school personnel, but it should be 
supplemented by focused field observations and follow-up studies on the 
basis of the survey results. 

6.6 Technologies^ Devices and Procedures 

This point concerning the distribution of research and develop- 
ment center technologies and products has been discussed in different 
contexts, (items 3.6 and U.6) 
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In this connection only the remitider is necessary that the studies 
of this distribution process must commence from both sides; by tracing the 
distribution from the center "outward," and by following the network of 
implementation and distribution from the schools "inward." 

6.T Manpower 

Certainly it is not necessary for the purpose of this impact 
study program to investigate the total manpower supply of American educa- 
tion. However, estimates of the characteristics and so\irces of school 

personnel providing those specialized services that make educational 
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research and development results available, would be useful. This item 
overlaps with the search for the sources and characteristics of manpower 
in the knowledge distribution and implementation system, but we refer 
here specifically to school personnel. Impacts on this manpower pool 
through contacts with the Research and Development Center Program and the 
laboratories, for example during training, through prior employment, and 
ethers, need to be considered. 
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7. INTERSYSTEM LINKS 
7 • 1 Major Questions 

The links between the four systems of knowledge production, 
distribution, implementation, and use are of particular importance for 
the purpose of tracing impacts of the knowledge production sector to the 
other ones. The question of the differential conductivity of the total 
system will receive a partial clarification from studies of linkages, which 
also can be thought of as system-system interfaces. 

In the empirical investigations, it might turn out that some of the 
system boundaries are diffuse, so that clearly defined linkages are dif- 
ficult to locate. It therefore may be tempting to restrict the investiga- 
tions to deliberately planned linkage or knowledge transfer arrangements. 
This should, however, not be done to the exclusion of studying '^natural," 
i.e., unplanned system interfaces, and linkages between system sectors 
that are affected by public media. 

The major questions in this area are: 

— What professional roles have developed in these linkage structures? 

— Are there gaps or role confusions? 

— What are the system interfaces, what specialized linkage structures 
have emerged J and how do they contribute to the conductivity of 
the total system connecting knowledge production and knowledge 

use? 
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7»2 Mapping the Domain and Its Structure 

The basic information for describing the array of linkage stru 
and system interfaces can be derived from the analysis of the four major 
systems of production, distribution, implementation, and use. The dif- 
ferences, for example, in standards, priority, organizational structures, 
describe the contexts for each of the system linkages. 

A special inventory of system interfaces is needed. This inventory 
would list concrete settings in which linkage structures arise at system 
interface points. Some such areas are sketched in ^he following items. 

m 

7.3 Knowledge Production-Distribution Links 

Arrangements r;pw exist for the distribution of research results 
to schol€UPly audiences, through publication and presentation of papers at 
meetings. One wonders > however, what the impact of these activities ip 
on the priorities and decisions of large publishers of educational 
materials. An importance question is whether the direct distribution of 
educational materials from the research and development system is more 
or less effective than indirect effects on the knowledge distribution system. 
Sources of these indirect effects include the setting or infl^^encing of 
priorities and of quality standards by those who are knowledge distribution 
gatekeepers. 

In this domain of system linkaget3 there arise a number of crucial 
legal questions, for example concerning copyright laws. They are connected 
to the economics of the distribution system and the manner in which 
decisions in it are made. A separate technical legal review of such norma- 
tive constraints is necessary in supplement to the impact study program. 
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Focused efforts to distribute research and developnent products 
to organized channels or networks ^ such as cooperative school networks » 
deserve particularly close attention ^ since some of them cut across all 
four systems and might afford useful sites of investigation. 

7 Knowledge Distribution-Implementation Links 

In this connection the particular requirements of the implementation 
process for knowledge and other supports come to mind. An illustration of 
this linkage structure can be foiind in the Follow-Through network^ and the 
studies conducted to ascertain the degree of implementation of certain 
programs. The characteristics of "implementors" have been mentioned 
as an importsuit topic of study in a different connection although little 
is known in a reliable manner about their interactions with the knowledge 
distribution and production systems. 

7,5 Knowledge Implementation-^Use Links 

The work of implementing a new program impinges on and- is con- 
strained by the use system. There is considerable literature on chsinge 
agents and innovators in schools but further work from the point of view 
of intersystem links is indicated. In particular, the question of long 
range affegts on the use system and the r ification of the intended 
program in becoming incorporated into the operating system needs to be 
raised. 
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7.6 Intersystem Links and the Directional Conductivity of the System . 

There are, of course, other system links, such as those directly 
connecting knowledge production and use, hut it seems pedantic and un«* 
necessary to list them all here. One point, however, needs to be emphasized 
about the total system of linkages. We have earlier used the concept of 
differential conductivity of this system for knowledge production and use, 
a concept which seems intuitively clear, in that different channels thf'dugh^ 
the system conduct different types of knowledge at different rates and 
with differentials in resistance and distortion. 

We now €uid the concept of directional conductivity, by which we 
mean that the flow of information may be very different from knowledge 
production to use, as. against, from the use system to knowledge production. 
For example, information about priority setting processes and constraints 
in different systems may become insufficiently available in the knowledge 
production sector. Further, information about the practical experience 
with innovations implemented and in use may not be made available to 
knowledge producers. Therefore, mechanisms for the multi-directional flow 
of infonnation, especially concerning priorities and knowledge needs, need 
to be investigated. The Research and Development Center Program itself, 
by emphasizing the need for product evaluation and evaluat' /e research of 
long range effects (at least in its intent ana program) may have impacted 
aspects of the multi-directional flow system in education. Clearly, such 
mult i -directional flow affects the capacity of the total system to be 
adapted and self-corrective. While we hesitate to recommend ity it might 
be instructive to attempt the construction of a model of the total system, 
its linkages and channels, in order to represent its differential con- 
ductivity characteristics. 
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8* STUDIES 

8.1 Introduction 

In this section we outline a program of studies designed to obtain 
the information demanded by the foregoing reflections. An important 
consideration is the comprehensive, synthetic nature of the information 
needs for mapping the system's structure, which contrasts with the nred 
for very fine grained and detailed measures in certain areas, especially 
in the study of linkages, implementation processes, and certain aspects 
of the knowledge production system. 

Obviously, the greater the precision of measurement, the greater 
the cost. Since some of the objects to be studied are of great size, the 
cost for high precision and fine grain measurements throughout the issues 
raised in the foregoing may be prohibitive. We therefore strive to combine 
rather coarse measurement methods and procedures for obtaining an overview 
of both a quantitative and qualitative nature, with selected procedures 
of "close-up" studies. The program of studies recommended here proceeds 
in terms of a multi-layered approach and combines historical and socio- 
logical methods with procedures of informed reportage. 

8.2 An Intellectual History of the R&D Centegr> Movement 

As one fundamental aspect of the impact study program, a^ monograph 
should be conmissioned which describes the background, development, and 
impact of the research and development center efforts on the field of 
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educational research, Its paradigms, standards, and its Intellectual 

■ 

productivity* It is thoiaght that this work may require one year of effort 
of a senior educational historian with an appropriate staff • 

Under item 3.5 (Stock of Knowledge in the Knowledge Production 
System) above, we suggested the exploration of the feasibility of stimulating 
a new assessment of the state of knowledge in the field, for example by the 
National Academy of Education. Such an effort, clearly, would be quite 
different from the historical study described here. There would, however, 
be overlaps, as there wo\xld be connections between the historical effort 
and the educational knowledge production study suggested below (see item 
8.7). Such partial overlaps are not likely to result in duplication of 
efforts, but in mutual reinforcement and are thus desirable. 

8 • 3 Review Symposia on Research and Development Policies in Education 

Three different review symposia by experts in knowledge distribution, 
educational research and development scholars, and leading practicing 
educators respectively are recommended. An assembly of fewer than 25 
persons for each topic, carefully selected and prepared by staff papers and 
a topical checklist, can be used to obtain important qualitative information 
on many of the items outlined above. Each symposium should be a working 
session of approximately two days. For example, the first symposium might con- 
sist of persons whose task it is to reflect, analyze and then to translate 
into the popular media trends and emphases in American education. 
Participants could be selected from five categories: (l) publis^hing 
houses, (2) major newspapers, (3) news magazines, (h) literary magazines, 
(5) other media. Examples of each of these categories might be the 
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following: A representative of the editorial board frcm three different 
kinds of publishing house s, those with strong lists in the area of 
children's fiction and non-fiction» publishers of college textbooks and 
publishers of texts for public school use. In addition t one might choose 
a house with a strong list of tradebooks relating to educa^^'-on. 

Obvious newspaper staffs to be represented in the person of their 
education editoi* might be the New York Times > the Washington Post > the 
Christian Science Monitor > the Wall Street Journal and the like. News 
magazines might include Time, Newsweek > U.S. News and World Report > 
also the special Sunday editors of the New York Times Magazine section and 
oihers. "Literary" magazines might include Saturday Review-World > Harper' s % 
Psychology Today t Rampart s > Atlantic Monthly . Finally one might think 
of television editors of teacher materials, including education editors 
on the major networks. 

We might begin by discussing the effectiveness of several policies 
and programs, ard of especially research and development efforts. More 
specific items can be selected from the outline suggested by the structure 
and analysis of this design paper which might serve as a guide towards 
stinicturing the effort.. 

In subsequent symposia leading educational research and development 
scholars, including representatives of the research and development program 
and of non-profit organizations, might address themselves to a similar 
set of questions and offer estimates of scope and nature of the R&D 
center impact. Finally, leading practicing educators, in a separate 
session, might be asked to assess the same domain. 
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It is essential, of course, that each^gyroposium be prepared by 
careful selection of participants, draft papers, and materials, and that 
competent staff be present to prepare a summary of observations and 
conclusions as a final report on each symposium. We feel that the procedure 
is very likely to yield important inputs into the design of studies to be 
carried on subsequently. 

Q.h Overviev Papers 

It is suggested that for the domains of knowledge distribution, 
implementation, and use separate overview papers be prepared. These 
papers are to draw. on available documentation in the NIE, formal and 
informal interviews, and published materials. The procedure envisioned is 
similar to that used by Francis S. Chase.^^ It is thought that each of these 
papers might require a half-year effort on the pajrt of an experienced person. 
The materials should become background for the design of more specific 
studies. 

8.5 Designs and Design Review Conference of Co oT^erating Investigators 

In order to achieve a reasonably integrated impact study program, 
it seems desirable to commit designs for the component studies at an early ' 
date and organize a design review conference in which all cooperating 
investigators participate. The objective of this conference is to review 
the plans for the research program, to utilize the result of the symposia 
and the overview papers, to receive a preliminary report on the progress 
of the historical work, and to arrive at specific decisions about special 
emphases and procedures in the research to be undertaken. 
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This step may appear unusual, but given the fact of the large scope 
of the enterprise and the need to achieve efficient coordination of efforts, 
it Ifi thought advisable to proceed in this fashion. 

8.6 School and School Districts Survey eind Field Studies of 
Knowledge' Utilization 

This large-scale project in the format of a major sample survey 
would be designed to investigate the topics outlined under section 6 in 
this paper (The Use System) and to provide linkage.s to the focused network 
studies and the case study program mentioned below. Capability should be 
provided for follow-up field work after the completion of the survey. 
Special emphasis might be made- on acquaintance with research axid development 
knowledge, its use, the nature of knowledge der^ands and priorities, in 
different school systems, knowledge search behavior, and the credibility 
of knowledge sources. 

One desired result would be the construction of indicators 
suitable for subsequent monitoring of the system. This must be thought 
of as a large effort which will require substantial resources. 

8.7 Educational Knowledge Production Study 

This is a recommendation for a sociology of science investigation 
of the items considered in section 5 of this paper (The Kr. owledge Pro- 
duction System). It must include trends in organizational patterns, 
standards, priorities, scope and nature of scientific production, networks 
of cooperation and citation, and nanpover aevelopment. Comparison with 



the state of affairs prior to the R&D center system ts possible, for. 
example through reference to the work of Sam Sieber and associates #^5 

This is a large scale effort which it might take one and a half 
years to complete. 

8.8 Focused Network Studies 

These are studies involving the detailed tracing of selected 
products through the network of contacts, commencing both from the centers 
outward and from the schools inward. They should involve attention to 
possible differences in patterns between early adopters and late adopters 
of the products chosen. Tracing the networks in complete detail is a 
cumbersome process; it therefore should only be undertaken with regard 
to selected products and their distribution. This effort should be linked 
both to the school survey and the knowledge production study. It will be 
necessary to estimate the most appropriate sequencing of these studies 
as a part of their design. 

8.9 Selected Linkage Case Studies 

Case studies are an appropriate device for investigating linkage 
arremgements. Several themes for such case studies have been suggested 
in section T of this paper (Intersystem Links). It would be desirable 
to construct a scheme outlining the domain so that different case studies 
can be more systematically evaluated with regard to the information 
yield for understanding the system. 
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8 . 10 Manpower Impact Studies' 

Throughout this paper emphasis has been given to the assessment 
of the Impact of the Resectrch and Development Center Program on manpower 
for knowledge production, distribution, and implementation specif ically^ 
The study of representative career patterns of educationed personnel having 
been in contact with research and development center or laboratory work 
at one stage in their career, as veil as studies of the characteristics and 
recruitment fields of personnel now working in the distribution and 
implementation systems are useful in this regard. 

8 . 11 Symposium of Princii^vl Investigators; Knowledge Production 
and Use in American Education 

In analogy to the design conference proposed for the initial 
phases ot this research program, it is suggested that it be coucluded 
by a symposium in which research report are presented, discussed, and 
an integrated ijiterpretation is attempted by the group of cooperating 
investigators. The recommendation is made because this procedure :iay 
facilitate the practical use of the knowledge gained. 

8.12 Construction of a Monitoring System of Educational Knowledg e 
Production and Use 

# 

The continuous observation or monitoring of a complex cultural 
institution is difficult at best and, if badly done, can have negative 
effects. Using such considerations such as the ones set out in this 
paper, a design for monitoring the system should be established simul- 
taneous with but separate from the design for this impact study program. 
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However, the monitoring measures should not be implemented until the reeulte 

of the study proposed here are available so that the experience gathered 

with this comprehensive investigation can be utilized for the Improvement 

of the monitoring system. For example, both the school survey and the 

knowledge production study 'may yield proposals for repeatable measurements. 

Similarly, the method of conducting symposia from time to time may be a 

* . 

gooJ device to supplement quantitative measures with qualitative insights. 
It is therefore suggested that an evaluation of the resulting research 
reports be conducced for the purpose of validating the design of the 
monitoring system. 
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Conclusions 

We have in this paper outlined a frajnework for a program of study 
through which the impact of ^he research and development center system on 
American education can be assessed* We have chosen a rather comprehensive, 
systemic approach, designed to elucidc.te the position of the Resecurch and 
Development Centers in the total knowledge production, distribution, 
implementation and use network. We believe that a comprehensive view of 
this kind, supplemented by specifically focused studies in critical areas, 
is necessary for a readistic assessment of the policy. It should be 
remembered that we are dealing here with a combination of governmental 
science and educationcuL policy, designed to impact a, major domain of 
American cxilture. No simple measures of cost effectiveness or quantity of 
productivity are a convincing substitute to the approach we propose. 

As a final note, we do wish to point out that the Federal policy 
under scrutiny and the concentrations in research vind devel<"pment it has 
produced are in many ways peculiarly American. Other countries, hcving 
different educational philosophies and systems, have proceeded rather 
differently, even though several of them have, under the stimulus of the 
American example, established somewhat similar institutions. It might be 
useful for policy makers to avail themselves of the advice of perceptive 
foreign scholars and educators, possibly by requesting them to offer 
critical reviews of the complex of reports and studies which might be 
the result of this effort. 
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